Surround view systems for passenger cars showing the vehicle from a bird's eye view are available from several manufacturers. A few solutions are also available for commercial vehicles, wherein multiple cameras are employed to cover various sides of the vehicle. These multiple cameras may produce partially overlapping views that should be combined (synthesized) in the overlap areas to obtain a combined image. Otherwise, the two separate views of the same object may confuse the driver.
Although the creation of the bird's eye view of a vehicle is a common driver assistance application, in the area where the views of two cameras are to be combined together, the visibility of the objects is still not yet optimal. Due to the projection of the camera images to the ground, vertical objects are projected in the bird's eye view image in a way that they appear along a line extending away from the camera (i.e. radially from the camera ground point). Hence, at the location where the views of two cameras meet, the objects are projected into the areas, which are visible from the other camera, but the other camera does not show the object at this position. This may lead to a disappearance of the object in the bird's eye view.
Different methods for synthesizing the images into a single bird's eye view are used by available systems. For example, a simple way is to separate the overlapping portions of the images with a straight line, and so to sidestep the problem, though at the cost of an abrupt change in the view as an object crosses the line. Another way is to allow an overlapping of the images, but to apply around the stitching line different kinds of blending procedures to obtain a transition from one camera view to the other.
Conventional blending procedures combine the images with a gradient transition from one image to the other by adding smoothly varying percentages of the different views in the overlapping area. This gradient blending area can be narrow (resulting in sharp transitions) or wide (resulting in slow transitions).
FIGS. 8A and 8B illustrate an example for a wide blending (in FIG. 8A) and an example for a narrow blending (in FIG. 8B).
In these examples a first camera 201 captures a first image 211 from an object 400 and a second camera 202 captures a second image 212 from the same object 400, but from a different perspective. Although the object 400 might be an upright standing person, in the first and second images 211, 212 the object 400 appears to extend away from the position of the first and second cameras 201, 202. Hence, the object 400 seems to extend in different directions. The length of the depicted object 400 depends on various quantities as e.g. the position of the first camera 201 (or the height differences between the camera position and person's height) and the distance of the object 400 from the first and second camera 201, 202.
In order to obtain a bird's eye view, the first image 211 and the second image 212 should be combined in a combined image 123. In this example the first image 211 and the second image 212 completely overlap so that the blending procedure should be applied for the whole area 213 covered by the first image 211 and the second image 212. A blending mask 230 is employed for the blending procedure, which interpolates smoothly from a white color (on the right hand side) to a black color (on the left hand side). The colors of the mask are associated with the two images or their respective image data. Therefore, the smooth transition from white to black corresponds to a smooth transition from one image to the other image.
In the blending mask 230, the second image 212 is assigned to the white color and the first image 211 is assigned to black color. When applying the blending (e.g. pixel by pixel), the combined image 123 depicts on the right-hand side only the second image 212. Therefore, on the right-hand side no object appears, because in the second image 212 the object is tilted toward the left. When continuing the blending toward the left-hand side, more and more contributions from the first image 211 are added (i.e. more pixel data from the first image 211 are added to pixel data of the second image 212). At the end, on the left-hand side of the combined image 123, only the first image 211 is depicted. Also at this position no object is visible, because it is tilted to the right in the first image 211. As a result, only in the intermediate region between the first and second camera 201, 202 a combination of the object 400 shown in the first image 211 and shown in the second image 212 is visible. Therefore, the combined image 123 seems to show two objects 400a, 400b, one object 400a from the contribution added from the second image 212 and another object 400b resulting from the object as depicted in the first image 211. It should be understood, however, these objects 400a, 400b are only copies of single object.
At the bottom of FIG. 8A an exemplary picture 240 is shown, wherein the two copies 400a and 400b of the object are visible. As described before, this doubling or ghost-like appearance of two copies (which actually relate only to one object) is a consequence of the different perspectives (angle of view) of the first and second cameras 201, 202 capturing images from different angles.
FIG. 8B depicts a combination of the same first and second images 211, 212, but now using a narrow blending instead of a wide blending employed in FIG. 8A. In the narrow blending the first image 211 and the second image 212 are interpolated in a more narrow blending region 235 and not over the whole width of the first and second image 211, 212 as done in FIG. 8A. Outside this blending region 235, either the first image 211 (on the left-hand side) or the second image 212 (on the right-hand side) are shown. As it is apparent from the positioning of the blending region 235, at least half of the object 400 is cut away so that only the leg-region of the exemplary person might be visible in the combined image 123. The head and body part will not be visible. Therefore, in the exemplary picture 240, depicted at the bottom of FIG. 8B, the body and head of the person are cut off so that the person is barely visible.
This is the aforementioned disappearance of parts of the object as consequence of a narrow blending. This, of course, causes confusion to the driver who does not know whether there is an obstacle or not.
Therefore, there is a demand for an image synthesizer, which overcomes the aforementioned problems and, in particular, generates a combined image that clearly depicts obstacles even close to the vehicle.